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Abstract 

In interactive media, it is the spaces between the text that are as important to read as the text itself. 
Visual literacy is a vital form of reading in an increasingly pictorial world. This paper examines the 
grammar of visual literacy by analysing the discourse of a group of adolescents and the way they 
interpret multi-layered, seriated and framed visuals from a personal, socio-cultural and structural 
perspective. The paper proposes the case for art education within a literacy context. 

Introduction 

Youth in Australia are bombarded with images. Being able to understand these images is 
a vital literacy. Images play a major role in understanding the world. The “information 
age” has led to the need to process volumes of data quickly and efficiently and the adage 
of “a picture being worth a thousand words” is revealed in the expressive way images are 
utilised in interactive media. From the moment we turn on a TV, computer or DVD, we 
are in the world of imagery (Matthews 1 997) 

From the days of earliest cave art, learning has been a visual process. People have always 
encountered visual images in their daily lives, yet in recent years the advent of digital 
technologies has led to a cognitive revolution in the perception of images (Sartorius 
1998). Visual information processing is a vital skill for people and involves more than 
simply being able to ‘see’ an image. Yet, limited research is being conducted into the 
cognitive processing and interpretation images. People need to be able to use, interpret, 
analyse and think critically about visual images and the significance of what they are 
seeing. 

Commercial designers study, conduct and apply visual literacy and communication 
research to product development, web pages and advertising. Extensive commercial 
research, such as the study conducted by Nike (Nike 2000), has been aimed at 
determining adolescents understanding and interpretation of visual imagery. While 
commercial bodies have realised the power of imagery and the need to be able to 
understand, and to manipulate children’s critiquing of these images, limited research has 
been conducted into this field. Jonasson (Jonassen 1996) argues that there is a need for 
research into the ways visuals and visualisation affect the learning process. Jonasson 
(Jonassen 1 996) sees visual literacy as being an eclectic notion, but when related to 
information technology it implies being able to think, learn and express oneself in terms 
of images. He argues that there is a need for further research that accentuates the way 
people construct meaning from visuals, especially in relation to the signs, symbols and 
icons presented in interactive multimedia. Yet he bemoans the fact that "multimedia has 
not yet become a research interest of visual literacists" (Braden in Jonassen 1996: 497). 
Jonasson feels that adolescents do not receive an adequate education in the critical 
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viewing of images and he states that there is an urgent need for good qualitative research 
that examines the manner in which electronic imagery relates to visual literacy. 

There is a blurring between traditional print literacies and what is now termed multi- 
literacies. Hypertext melds text with images and sounds, and expands views of 
communication. Text has become very much a visual construction, which is non-linear. 
The impact of this new visual technology is most apparent in children in the middle years 
of school. In excess of 90% of children in the 10-15 years age bracket have access to the 
Internet (Tapscott 1998: 3). Middle school aged children have grown up in a world 
surrounded by interactive media. They use technology to play, learn, communicate and 
form relationships (Tapscott 1998:7). Teenagers represent the first group in Australia to 
have grown up surrounded by digital media and learning. Tapscott (Tapscott 1998: 151) 
describes this as the ascendancy to power of the “net generation” (“N-gen”). These 
adolescents do not only consume media, but they are constantly learning how to create 
imagery as well. The vast majority of adolescents report knowing how to use a 
computer, with nearly everyone having direct experience with video games and personal 
computers (Tapscott 1998:3). Interactive multimedia is widely used in schools and it is 
increasingly a tacit expectation that adolescents understand the purpose of imagery, can 
interpret images, are able to determine relationships among images and read layering 
within images. Yet there is a lack of research into the way visual images are interpreted 
and function within the minds of adolescents. Adolescents may appear to be able to 
superficially read the imagery thrust upon them, but often they remain unaware of the 
deeper layers contained deliberately and incidentally in visual communication. 

Visual literacy skills are vital for both high school and university students to be able to 
read and interpret the material presented to them (Christopherson 1996; Osborne 1998; 
Roblyer 1998; Hubbard 1999). New literacies are being generated that will shape future 
information and knowledge cultures (Garton 1997: 209). Leh’s (Leh 1999) study in the 
United States of America, observing adolescents' preferences when viewing and 
developing web sites found that children were particularly attracted to web sites with 
many graphics or animations, and that girls and boys used very different visual images 
for their Web page development. 

Technology related communication is highly complex and dynamic. Information 
technology has simultaneously become more sophisticated in its application, while 
becoming more simplified and accessible in its use. Using interactive media everyone 
becomes both producer and participant in the visual culture. Adolescents live in a world 
where visuals are increasingly integrated into the media. The Web, video clips, CD 
Roms and television are all intensely graphical environments. There has been a 
proliferation of non-print media, which is interactive and generative. As our society 
moves forward in the information age, strategies for learning from various forms of texts 
play an increasingly important role in our conception and definition of what it means to 
be fully literate. As Smith (Smith 1997: 1) indicates “Images are now used primarily as 
shorthand methods of representing information... increasingly symbolic orientation of 
communication compels the writing instructor to address the dynamics of visual 
discourse.” The world of the Internet is a fabricated world that has been created to be 
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experienced and interpreted in a particular way. It is a simulated environment that relies 
heavily upon the power of visual persuasion. Information presented using interactive 
multimedia is not read in a traditional sense, but rather explored holistically and 
divergently. 

The study 

This paper reports on an investigation into adolescents' interpretations of visual 
communication that has commenced in Australia. The aim of the study is to use 
interactive technology to survey adolescents on their visual and written responses to a 
range of images. It is anticipated that the results of this study will provide detailed 
information on how to interpret and develop images for use in interactive media and lead 
to the identification and character of visual literacy skills for the future. 

The study aims to determine how adolescents interpret visuals in their use of interactive 
media and to provide understanding and guidelines for visual literacy and interactive 
technologies. This study focuses on adolescents, who are the first generation to grow up 
and be educated in a interactive multimedia environment. This study provides a forum in 
which students may produce knowledge in web based visual responses. The visual 
literacy constructs will be identified and defined from the discourse of the participants. 
The adolescent’s own conversations about images will be recorded via the Internet and be 
used to formulate themes for addressing visual literacy. The study takes a diverse and 
multi-referential view of literacy and involves teachers and adolescents as active 
collaborators in the research process. 

Using interactive technology, this project aims to identify the cognitive tools used in 
visual communication by exploring how adolescents interpret imagery, construct 
meaning and apply new technologies. Adolescents are surveyed via images presented 
through a website and respond to questions and images in text and visual image 
manipulation. The methodology involves developing a World Wide Web site that 
provides teachers and students with a range of visual images to be responded to and 
commented upon. Using discussion forums and interactive online drawing boards, 
teachers and students are encouraged to discuss issues online in both written and visual 
form. Commentary and images are stored in an online linked database enabling the 
researchers to deconstruct and explore emerging themes. The use of new technologies in 
the research methodology, allows ease of accessibility for project participants from 
geographically and culturally diverse groups. Importantly, the site allows flexibility for a 
truly national research project, including remote geographic locations, while providing a 
unique interactive means for communicating the results of the study to all the researchers 
and participants involved. Findings will be reported directly back to participants and the 
broader community using web-based technology, and participants will be encouraged to 
constantly critique and reflect upon their comments and input into the study. 

Results 

While still in the design phase, it is anticipated that the results, gathered over two years, 
will provide a better understanding of the cognitive processing and affordances in relation 
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to visual imagery on the World Wide Web, through the deconstruction of visual and 
responsive text interpretations by adolescents. 

The study is based on the premise that communication is not set in time, but rather is 
constantly modified as it operates in resonance with people and the environment. It is 
assumed that online communication involves the development of new aesthetics and 
grammatical genres in response to the changing communication structures. The 
structures, which emerge from acts of communication, serve as frameworks for the 
investigation of inter-ethnic and inter-cultural understandings in relation to various 
communication forms. This project will address these areas by: 

1. Highlighting significant national trends by targeting representative schools and 
adolescents in all states and territories within Australia. A range of student 
responses will be sought from a variety of locations, socio-economic areas and 
genders; and 

2. Experimenting and leading with new technologies as a basis for research and 
information retrieval, by developing a world wide web site which will act as both 
a communication device between researchers and participants and a retrieval 
device for the information collected and collated. 

The results of this study should encourage educationalists to develop strategies for the 
enhancement of visual communication skills in young people. This is an area of great 
need, as the world becomes more visual, and as John (John 1997) noted, “in terms of this 
new visual literacy, [current] education produced illiterates”. The research is structured 
in such a way as to promote the exploration of connections between socio-cultural and 
gender backgrounds and the way images are interpreted and engender discussion of the 
relationship between pictures and reality. Current technology allows for the manipulation 
of imagery. Students need to be familiarised with the manipulation that is inherent in 
visual communication. There is a need to re-evaluate the “truth value placed on images 
and the importance of a higher level of visual literacy in order to better identify image 
manipulation.” (Sartorius 1998: 1). Students and teachers need to be aware of the 
potential for misuse and unethical handling of images. The study explores the 
construction and interpretation of broader issues of literacy and brings images into the 
classroom for discussion. The interactive website will be used to explore, critique and 
assess of the impact of multiple literacies in Australian middle schools. 

Discussion 

Visual communication in an interactive environment is constructed of multi-layered and 
ambiguous symbols systems that are both syntactically and semiotically dense. In visual 
communication, the syntactic elements are those visual characters that are used, while the 
semantic elements relates to the correlation, compliance and connection between the 
visual symbols that serves to deposit meaning into the visual metaphors used to 
communicate. These symbol systems are highly complex and dynamic. It is a near 
impossible task to document a comprehensive taxonomy of the syntax and semantics of 
visual representations. Artists have been trying to understand visual communication for 
many centuries, and while visual language continues to expand, it could be argued that 
we are not getting any closer to fully understanding the nature and power of the visual. 
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Frames of viewing 



The world we see online depends upon how it is framed. Frames are not stagnant 
entities, but rather shifting structures that serve to organise meaning. Juxtaposition and 
ambiguity are used to forge connections between seemingly obscure things. In this 
sematic fusion, the introduction of a context not normally associated with an image 
serves to establish new target domains, causing the reader to view the original image in a 
different way. Images as with words can be ambiguous, and it is this ambiguousness that 
is explored through the development of syntactical frames. These syntactically frames 
can be gained through proximity of images, chunking, clustering, continuation or 
sequence of images, establishing a common region, or through the manipulation of prior 
or current associations. Each of these strategies works to encourage the brain to combine 
elements to create integrated meaning. Lester (Lester 1995: 123-127) identified six ways 
that could be used to frame the systematic critique of visual communication. These 
include personal, historical, technical, ethical, cultural and critical readings. 

The personal frame provides a subjective method of critique. It is the 'gut reaction' to an 
image. It is based on individual feelings and is often characterised by strong reactions 
that can be manifest as either likes or dislikes. While not essentially culturally bound, it 
is likely that personal readings of imagery reflect inherent, and often tacit, cultural 
assumptions and predispositions. 

Historical reading of imagery is common practice in traditional fine arts, yet it is largely 
ignored in students' reading of interactive multimedia. Despite a relatively short history, 
digital image making and interpretation have clear historical contexts. The technical and 
cultural development of multimedia gave rise to circumstances that have profoundly 
influenced the visual metaphors we use. The 'apple' is an example of a historically bound 
symbol that obtains its meaning not from its physical appearance but rather from the 
historical context of that icon. Similarly, the desktops and folders, are not in themselves 
metaphors, but rather the imagery contained in this symbol is bound by the history behind 
the development of these visual representations. The term 'surfing' has adopted a life 
independent of its history and shows the way in which a visual or envisioning metaphor 
is both grounded in history and develops a life independent of history. 

Technically, there is a complex grammar of visual representation to be studied from both 
the receptive and productive frame. Students need to be aware of how an image has been 
produced. With a clear understanding of the techniques involved in generating online 
images, students are better placed to be able to know when production values are high 
and low. Students should discuss issues of quality and be aware of the ways images are 
constructed and modified in interactive media. Increasingly, the media will also 
encourage students to respond visually, and they will require the skills and techniques to 
produce visual responses. 

The challenge of ethics in the way students read images in interactive media is an 
educational imperative. Where images are primarily used to persuade and subvert, 
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students require an awareness of the potential of such power. The easy ability to be able 
to superficially react to images belies the complexity contained within the construction of 
visuals. Students should be educated to critically reflect on the theoretical dimensions of 
the way images are presented. 

The meaning of the symbols used in visual communication is bound by the culture in 
which they were created. In an interactive environment, this goes beyond the mere 
content of the image. Cultural impacts are manifest in the narrative structure of the 
visual, the style of the artwork, the selection of words to accompany the image, and the 
attitudes and cultural statements made by the work. The semiotic reading of the image is 
largely dependent upon the cultural community of the artist and the perceived, or desired, 
culturally mores of the viewers. 

Being able to read online visuals critically involves being able to transcend a particular 
image to infer generalisations and judgements about the medium, the culture from which 
it is produced and ultimately the viewer. Images need to be critiqued to reveal the 
linkages between the image produced and the society that accepts or rejects these images. 
This can be a challenging and confronting process for the individual as it involves 
redefining personal value structures in the light of reflections on the human condition and 
more universal understandings. 

Grammatical structures within the frames 

Within each of the aforementioned frames through which critiques of visuals can be 
made, there are a number of grammatical structures which are connected to the frames, 
but operate directly on human perception and interpretation of visuals. 

Humans naturally have a propensity to group like images together and to make 
connections between seemingly disparate imagery. For example, when the picture of a 
car is juxtaposed with the picture of the contented and gorgeous family, the viewer will 
automatically connect these two ideas, to develop a concept that the car and family exist 
together as a picture of prestige, safety and personal status. This does not need to be 
explained, as the presence of the enclosing border in the advertisement, serves to bind 
these images into single unit of meaning. There is a natural desire to align visuals into a 
smooth directional continuation, so even without the defining visual frame, people will 
seek spaces to form this connectiveness. This is very apparent in interactive media, 
where images are placed in rapid succession to create a 'feel' or emotional connectiveness 
with the space that has been created on screen. Interestingly, as pop music video clips 
show very effectively, even when visual stimuli are highly ambiguous, human beings will 
attempt to make perceptions as "good" as the images allow. People create unity to give 
the images a sense of closure. This human capacity is exploited in interactive media as it 
allows the producer of images to play with the intangible to get desired ideas to emerge 
or be ratified. 

Throughout time, artists have been aware of the symbolic power of images. It has also 
been realised that a number of visual devices interact to produce the strongest possible 
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communicative power in visual representation. All visual symbols contain value. Value 
may be actual, such as a striking red flag placed against and blue sky, or conceptual, 
such as a child peering through the hole in a wire gate. Actual value is established 
through a number of visual conventions such as texture, balance, detail, colour and size. 
Increasingly in interactive media, conceptual value is achieved through the addition of a 
number of other components that complement the aesthetic ways in which actual value is 
visually presented. These components are loosely called 'grammar' as they impact on the 
arrangement of images and the way the visual symbols are read and interpreted in an 
interactive environment. As this form of communication is relatively new and still 
evolving, similarly the grammars described reflect only a fraction of possible 
interpretations of online visual literacy. These grammars are predominantly seen in 
interactive and multi media, but may also exist in a different form in more traditional 
forms of art. For example, layering is a visual 'grammar' of online images. In a 
technical sense it means the placing of repeated transparencies over an image that change 
the original representation. Initially, these processes were done to re-contextualise or 
enhance an image, but increasingly now it is done to modify images. The application of 
layers gives both visual thickness and conceptual depth to the original or source image. 

Motion and location in space are significant grammars of interactive visual literacy. 
Increasingly, interactive designers and artists are moving away from the metaphor of the 
computer screen as a blank canvas. It is no longer regarded as a 2Dimensional space to 
be filled, but rather interactive artists see locations in space as constructed and 
manipulable. Objects float randomly in a limitless space created visually by entering 
multiple coordinates. At the push of a button, flat shapes can be turned into 3- 
dimensionsal forms that move within a created space. The 'wire frame' allows for the 
creation of images that can walk, talk and react. Vision is no longer simply a matter of 
reading a still image in a flat plane. Codes of movement and energy are the characteristic 
of interactive visuals. Motion and reaction are a significant part of interpreting online 
images. 

Seriation is the manner in which multiple copies of images can be generated saved and 
manipulating. Unlike traditional art forms, when working in interactive graphics, it is 
possible for artists to maintain an infinite number of variations on a piece of work. While 
art galleries conservers marvel at x-rays of paintings under paintings or preliminary 
sketches of paintings under oil paint, interactive images exist as multiples. This is 
significant as seriation of images serves to question notions of originality and 'process 
versus product' debates in visual communication. With seriation, at any stage the making 
process can be the product (even retrospectively) and vice versa. Similarly, where one 
image exists, millions of copies can be made at the touch of a print key. 

To understand and read on screen images the students also need to be aware of the role of 
illumination in image construction. The image we see on the computer screen, TV 
screen or DVD, is not based on paint and pigment. It exists only virtually as light 
sources. It is the play of illumination that allows our eyes to see the image. As it does 
not rely on set pigments, it is a transient and personalised. The physical perception of 
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your eyes will determine what you see and this will be different from what others see 
looking at the same collection of reflected light. 

Resemblance is a significant factor in reading visual images in interactive media. 
Resemblance in the online environment, is not about making a thing look real, rather it is 
about the use of abstraction for symbolic and or metaphoric purposes. Similarity 
becomes more powerful than imitation. The resemblance process is made easy through 
the use of graphical manipulation tools, which allow the artist to make choices about the 
way images will be presented, as opposed to represented. Connected with the process of 
semblance, is the use of proximity and connectedness to give the images a sense of 
closure and a stronger symbolic power. For example, the image of a woman in a short 
skirt stepping out of the car has been generated not to represent women, but to 
symbolically present sexual imagery. The adding of a whisky bottle in the same frame as 
the long legged women then further strengthens the message, and the viewer makes the 
connection that these fleeting images communicate that whisky is sexy for men to drink. 

Connected with resemblance, simplification is a feature of image reading in a 
multimedia environment. Time is precious and so unnecessary detail and contextual 
decoration is peeled away. Only enough detail is provided to make the image accessible. 
Too much detail makes the work too time consuming to read. The people employing the 
persuasive strategies do not want people to dwell too long over the image. The idea is 
that the simplified, symbolic image, makes its point, is read and connected with in 
microseconds. Longer engagement is discouraged as it provides too much time for 
consideration and to think. 

Text in interactive visuals is kept to the barest minimum. It is used as a design device or 
where visual reading needs some further support to reduce ambiguities and to increase 
communicative power. Often only one or two words are used, and it is the size and space 
of the text that is as important as the meaning of the text itself. Words are associated 
with visual elements to communicate related information and to clarify the meaning and 
interpretation of each other. Labelling is frequently used in an interactive environment 
to assist in the process of forming visual relationships between or within symbolic 
elements. 

Foregrounding and space are significant in the way meaning is derived in an interactive 
environment. By foregrounding some aspects and eliminating others, certain readings are 
given prominence. When space is left in an image, there is a natural desire to complete 
an idea established by a visual. The process of synecdoche (Horn 1998), where a part of 
an image can be used to represent the whole, and metonymy (Horn 1998), where one 
thing is used for another for which it is associated, are powerful forces in creating 
persuasive online imagery. The human eye and mind has an enormous ability to gather 
scattered fragments of images into a cohesive and coherent whole. 

The use of metaphor is a particularly strong characteristic of online visual 
representations. While metaphors exist in all forms of communication, visual metaphor 
is used predominantly in interactive media as a persuasive device. Political and social 
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metaphors are applied to images to make discourse and communication explicit. The 
'global village', 'virtual reality', 'desktops' and 'folders' are just some of the metaphors that 
define the online environment. The computer screen is largely visually encountered 
through a series of visual metaphors. The metaphors of the online world are constructed 
by combining visual and textual symbols. These symbols constitute the instrument used 
in interactive media to interpret the world, and images become the tool by which 
sensation is transferred into readable and shared experiences. The symbols in traditional 
print forms are largely abstract and conventionalised to its referent, while in electronic 
imagery, the symbol systems are increasingly iconic, specific and layered. The symbols 
are dependent upon a range of contextual, non-verbal and attitudinal referents (Gozzi 
1999: 142) to construct and reconstruct meaning. Interactive visual symbols are 
predominantly attributive and suggestive. They rely on mood and atmosphere to carry 
their message. They are affectively tinged pictures that aim at direct engagement through 
our emotions. The symbols are not sequential, and tend to deliberately throw you into the 
action somewhere after it has started. Final readings are not possible, and conclusions 
tend to be arrived at, rather than shown. In interactive media, many facts of each symbol 
may be presented simultaneously. All the pieces are given at once, and the viewer 
interprets these, not as a representation of reality but as a presentation of emotion and 
importance. 

Conclusion 

To conclude, students need to understand the structures underpinning visual presentation 
in an online environment. They require an understanding of the nature of new and 
emerging grammatical structures and the way these apply to the reading of images in an 
interactive environment. This is particularly imperative because visual communication is 
ultimately an act of persuasion and social discourse. Advertisers are unashamedly aware 
of the potential for use, and abuse, of visual codes and pictorial metaphors. As the 
Benetton campaign suggests, almost any image is justified if it get the attention of the 
potential customers. Visual persuasion is largely accepted without question and little 
consideration is given to the way in which images serve to change individual's attitudes. 
The power of imagery is not unique to interactive media. Throughout time images have 
been used to both help and harm others (Lester 1995: 73). Pictures are emotionally 
powerful forms of communication. Images in interactive media are used to persuade and 
to perpetuate ideas that words cannot adequately elucidate. Students need to be 
encouraged to be aware of how to view and produce visual images. They should be made 
aware of the manner in which pictures are used to convince others of a certain point of 
view. While in the early stages, it is hoped that the study of adolescents' discourse in 
regard to online imagery will provide ways forward for educators to 'teach' the 'reading' 
of imagery in an increasingly visual society. 
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